Improving Search through A3C Reinforcement Learning based Conversational Agent

We develop a reinforcement learning based search assistant which can assist users through a set of actions and sequence of interactions to enable them realize their intent. Our approach caters to subjective search where the user is seeking digital assets such as images which is fundamentally different from the tasks which have objective and limited search modalities. Labeled conversational data is generally not available in such search tasks and training the agent through human interactions can be time consuming. We propose a stochastic virtual user which impersonates a real user and can be used to sample user behavior efficiently to train the agent which accelerates the bootstrapping of the agent. We develop A3C algorithm based context preserving architecture which enables the agent to provide contextual assistance to the user. We compare the A3C agent with Q-learning and evaluate its performance on average rewards and state values it obtains with the virtual user in validation episodes. Our experiments show that the agent learns to achieve higher rewards and better states.Comment: 17 pages, 7 figure

Integrating evolution into ecological modelling: accommodating phenotypic changes in agent based models.

PMCID: PMC3733718This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Evolutionary change is a characteristic of living organisms and forms one of the ways in which species adapt to changed conditions. However, most ecological models do not incorporate this ubiquitous phenomenon. We have developed a model that takes a 'phenotypic gambit' approach and focuses on changes in the frequency of phenotypes (which differ in timing of breeding and fecundity) within a population, using, as an example, seasonal breeding. Fitness per phenotype calculated as the individual's contribution to population growth on an annual basis coincide with the population dynamics per phenotype. Simplified model variants were explored to examine whether the complexity included in the model is justified. Outputs from the spatially implicit model underestimated the number of individuals across all phenotypes. When no phenotype transitions are included (i.e. offspring always inherit their parent's phenotype) numbers of all individuals are always underestimated. We conclude that by using a phenotypic gambit approach evolutionary dynamics can be incorporated into individual based models, and that all that is required is an understanding of the probability of offspring inheriting the parental phenotype

New application of intelligent agents in sporadic amyotrophic lateral sclerosis identifies unexpected specific genetic background

<p>Abstract</p> <p>Background</p> <p>Few genetic factors predisposing to the sporadic form of amyotrophic lateral sclerosis (ALS) have been identified, but the pathology itself seems to be a true multifactorial disease in which complex interactions between environmental and genetic susceptibility factors take place. The purpose of this study was to approach genetic data with an innovative statistical method such as artificial neural networks to identify a possible genetic background predisposing to the disease. A DNA multiarray panel was applied to genotype more than 60 polymorphisms within 35 genes selected from pathways of lipid and homocysteine metabolism, regulation of blood pressure, coagulation, inflammation, cellular adhesion and matrix integrity, in 54 sporadic ALS patients and 208 controls. Advanced intelligent systems based on novel coupling of artificial neural networks and evolutionary algorithms have been applied. The results obtained have been compared with those derived from the use of standard neural networks and classical statistical analysis</p> <p>Results</p> <p>Advanced intelligent systems based on novel coupling of artificial neural networks and evolutionary algorithms have been applied. The results obtained have been compared with those derived from the use of standard neural networks and classical statistical analysis. An unexpected discovery of a strong genetic background in sporadic ALS using a DNA multiarray panel and analytical processing of the data with advanced artificial neural networks was found. The predictive accuracy obtained with Linear Discriminant Analysis and Standard Artificial Neural Networks ranged from 70% to 79% (average 75.31%) and from 69.1 to 86.2% (average 76.6%) respectively. The corresponding value obtained with Advanced Intelligent Systems reached an average of 96.0% (range 94.4 to 97.6%). This latter approach allowed the identification of seven genetic variants essential to differentiate cases from controls: apolipoprotein E arg158cys; hepatic lipase -480 C/T; endothelial nitric oxide synthase 690 C/T and glu298asp; vitamin K-dependent coagulation factor seven arg353glu, glycoprotein Ia/IIa 873 G/A and E-selectin ser128arg.</p> <p>Conclusion</p> <p>This study provides an alternative and reliable method to approach complex diseases. Indeed, the application of a novel artificial intelligence-based method offers a new insight into genetic markers of sporadic ALS pointing out the existence of a strong genetic background.</p

Evolution with Stochastic Fitness and Stochastic Migration

Migration between local populations plays an important role in evolution - influencing local adaptation, speciation, extinction, and the maintenance of genetic variation. Like other evolutionary mechanisms, migration is a stochastic process, involving both random and deterministic elements. Many models of evolution have incorporated migration, but these have all been based on simplifying assumptions, such as low migration rate, weak selection, or large population size. We thus have no truly general and exact mathematical description of evolution that incorporates migration.We derive an exact equation for directional evolution, essentially a stochastic Price equation with migration, that encompasses all processes, both deterministic and stochastic, contributing to directional change in an open population. Using this result, we show that increasing the variance in migration rates reduces the impact of migration relative to selection. This means that models that treat migration as a single parameter tend to be biassed - overestimating the relative impact of immigration. We further show that selection and migration interact in complex ways, one result being that a strategy for which fitness is negatively correlated with migration rates (high fitness when migration is low) will tend to increase in frequency, even if it has lower mean fitness than do other strategies. Finally, we derive an equation for the effective migration rate, which allows some of the complex stochastic processes that we identify to be incorporated into models with a single migration parameter.As has previously been shown with selection, the role of migration in evolution is determined by the entire distributions of immigration and emigration rates, not just by the mean values. The interactions of stochastic migration with stochastic selection produce evolutionary processes that are invisible to deterministic evolutionary theory

The Real maccoyii: Identifying Tuna Sushi with DNA Barcodes – Contrasting Characteristic Attributes and Genetic Distances

BACKGROUND:The use of DNA barcodes for the identification of described species is one of the least controversial and most promising applications of barcoding. There is no consensus, however, as to what constitutes an appropriate identification standard and most barcoding efforts simply attempt to pair a query sequence with reference sequences and deem identification successful if it falls within the bounds of some pre-established cutoffs using genetic distance. Since the Renaissance, however, most biological classification schemes have relied on the use of diagnostic characters to identify and place species. METHODOLOGY/PRINCIPAL FINDINGS:Here we developed a cytochrome c oxidase subunit I character-based key for the identification of all tuna species of the genus Thunnus, and compared its performance with distance-based measures for identification of 68 samples of tuna sushi purchased from 31 restaurants in Manhattan (New York City) and Denver, Colorado. Both the character-based key and GenBank BLAST successfully identified 100% of the tuna samples, while the Barcode of Life Database (BOLD) as well as genetic distance thresholds, and neighbor-joining phylogenetic tree building performed poorly in terms of species identification. A piece of tuna sushi has the potential to be an endangered species, a fraud, or a health hazard. All three of these cases were uncovered in this study. Nineteen restaurant establishments were unable to clarify or misrepresented what species they sold. Five out of nine samples sold as a variant of "white tuna" were not albacore (T. alalunga), but escolar (Lepidocybium flavorunneum), a gempylid species banned for sale in Italy and Japan due to health concerns. Nineteen samples were northern bluefin tuna (T. thynnus) or the critically endangered southern bluefin tuna (T. maccoyii), though nine restaurants that sold these species did not state these species on their menus. CONCLUSIONS/SIGNIFICANCE:The Convention on International Trade Endangered Species (CITES) requires that listed species must be identifiable in trade. This research fulfills this requirement for tuna, and supports the nomination of northern bluefin tuna for CITES listing in 2010

Lensing without borders. I. A blind comparison of the amplitude of galaxy-galaxy lensing between independent imaging surveys

Lensing Without Borders is a cross-survey collaboration created to assess the consistency of galaxy-galaxy lensing signals (ΔΣ) across different data-sets and to carry out end-to-end tests of systematic errors. We perform a blind comparison of the amplitude of ΔΣ using lens samples from BOSS and six independent lensing surveys. We find good agreement between empirically estimated and reported systematic errors which agree to better than 2.3σ in four lens bins and three radial ranges. For lenses with zL &amp;gt; 0.43 and considering statistical errors, we detect a 3-4σ correlation between lensing amplitude and survey depth. This correlation could arise from the increasing impact at higher redshift of unrecognised galaxy blends on shear calibration and imperfections in photometric redshift calibration. At zL &amp;gt; 0.54 amplitudes may additionally correlate with foreground stellar density. The amplitude of these trends is within survey-defined systematic error budgets which are designed to include known shear and redshift calibration uncertainty. Using a fully empirical and conservative method, we do not find evidence for large unknown systematics. Systematic errors greater than 15 per cent (25 per cent) ruled out in three lens bins at 68 per cent (95 per cent) confidence at z &amp;lt; 0.54. Differences with respect to predictions based on clustering are observed to be at the 20-30 per cent level. Our results therefore suggest that lensing systematics alone are unlikely to fully explain the ‘lensing is low’ effect at z &amp;lt; 0.54. This analysis demonstrates the power of cross-survey comparisons and provides a promising path for identifying and reducing systematics in future lensing analyses

Mesenchymal stem cells: from experiment to clinic

There is currently much interest in adult mesenchymal stem cells (MSCs) and their ability to differentiate into other cell types, and to partake in the anatomy and physiology of remote organs. It is now clear these cells may be purified from several organs in the body besides bone marrow. MSCs take part in wound healing by contributing to myofibroblast and possibly fibroblast populations, and may be involved in epithelial tissue regeneration in certain organs, although this remains more controversial. In this review, we examine the ability of MSCs to modulate liver, kidney, heart and intestinal repair, and we update their opposing qualities of being less immunogenic and therefore tolerated in a transplant situation, yet being able to contribute to xenograft models of human tumour formation in other contexts. However, such observations have not been replicated in the clinic. Recent studies showing the clinical safety of MSC in several pathologies are discussed. The possible opposing powers of MSC need careful understanding and control if their clinical potential is to be realised with long-term safety for patients

Iron Behaving Badly: Inappropriate Iron Chelation as a Major Contributor to the Aetiology of Vascular and Other Progressive Inflammatory and Degenerative Diseases

The production of peroxide and superoxide is an inevitable consequence of aerobic metabolism, and while these particular "reactive oxygen species" (ROSs) can exhibit a number of biological effects, they are not of themselves excessively reactive and thus they are not especially damaging at physiological concentrations. However, their reactions with poorly liganded iron species can lead to the catalytic production of the very reactive and dangerous hydroxyl radical, which is exceptionally damaging, and a major cause of chronic inflammation. We review the considerable and wide-ranging evidence for the involvement of this combination of (su)peroxide and poorly liganded iron in a large number of physiological and indeed pathological processes and inflammatory disorders, especially those involving the progressive degradation of cellular and organismal performance. These diseases share a great many similarities and thus might be considered to have a common cause (i.e. iron-catalysed free radical and especially hydroxyl radical generation). The studies reviewed include those focused on a series of cardiovascular, metabolic and neurological diseases, where iron can be found at the sites of plaques and lesions, as well as studies showing the significance of iron to aging and longevity. The effective chelation of iron by natural or synthetic ligands is thus of major physiological (and potentially therapeutic) importance. As systems properties, we need to recognise that physiological observables have multiple molecular causes, and studying them in isolation leads to inconsistent patterns of apparent causality when it is the simultaneous combination of multiple factors that is responsible. This explains, for instance, the decidedly mixed effects of antioxidants that have been observed, etc...Comment: 159 pages, including 9 Figs and 2184 reference